Make iconv //IGNORE behavior conform to POSIX and the docs #16846
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
While working on #16840, I discovered that an old glibc bug workaround is causing our
iconv()
implementation to go against POSIX and its own docs.There is a very detailed commit message on the commit that removes the workaround, but in short: the workaround ignores invalid input sequences, when what it should have been doing is ignoring sequences that have no representation in the target encoding. But the need for a workaround is also not detected/applied correctly.
(It would be nice if PHP could abstract the
//IGNORE
and//TRANSLIT
magic away from the underlying iconv, but now that POSIX 2024 includes them, the smart thing to do is just wait.)Afterwards, there's a test that's testing for the incorrect output. We fix & rename it.